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Foreword 

This Technical Specification (TS) has been produced by ETSI 3rd Generation Partnership Project (3GPP). 

The present document may refer to technical specifications or reports using their 3GPP identities, UMTS identities or 
GSM identities. These should be interpreted as being references to the corresponding ETSI deliverables. 

The cross reference between GSM, UMTS, 3GPP and ETSI identities can be found under 
http ://webapp . etsi.org/kev/queryform. asp . 
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Foreword 



id , 



This Technical Specification has been produced by the 3 Generation Partnership Project (3GPP). 

The contents of the present document are subject to continuing work within the TSG and may change following formal 
TSG approval. Should the TSG modify the contents of the present document, it will be re-released by the TSG with an 
identifying change of release date and an increase in version number as follows: 

Version x.y.z 

where: 

x the first digit: 

1 presented to TSG for information; 

2 presented to TSG for approval; 

3 or greater indicates TSG approved document under change control. 

y the second digit is incremented for all changes of substance, i.e. technical enhancements, corrections, 
updates, etc. 

z the third digit is incremented when editorial only changes have been incorporated in the document. 
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Scope 



The present document specifies test procedures and digital test sequences to be used for conformance testing of 
implementations of the Extended Adaptive Multi-Rate Wideband (AMR-WB+) codec. 



References 



The following documents contain provisions which, through reference in this text, constitute provisions of the present 
document. 

• References are either specific (identified by date of publication, edition number, version number, etc.) or 
non-specific. 

• For a specific reference, subsequent revisions do not apply. 

• For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP document (including 
a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same 
Release as the present document. 

[1] 3GPP TS 26.290: "Audio codec processing functions; Extended Adaptive Multi-Rate - Wideband 

(AMR-WB+) codec; Transcoding functions". 

[2] 3GPP TS 26.304: - "Extended Adaptive Multi-Rate - Wideband (AMR-WB+) codec; 

Floating-point ANSI-C code". 

[3] 3GPP TS 26.273: "ANSI-C code for the fixed-point Extended Adaptive Multi-Rate - Wideband 

(AMR-WB+) speech codec". 

[4] ITU-R Recommendation BS. 1387-1: "Method for objective measurements of perceived audio 

quality". 

[5] 3GPP TS 26.204: "ANSI-C code for the floating-point Adaptive Multi-Rate - Wideband (AMR- 

W) speech codec". 

[6] 3GPP TS 26. 1 74: "AMR speech codec, wideband; Test sequences" . 



3 Definitions and abbreviations 

3.1 Definitions 

For the purposes of the present document, the terms and definitions given in 3GPP TS 26.290 [1], 3GPP TS 26.304 [2] 
and 3GPP TS 26.273 [3] apply. 

3.2 Abbreviations 

For the purposes of the present document, the following abbreviations apply: 

ACELP Algebraic Code Excited Linear Prediction 

AMR Adaptive Multi-Rate 

AMR-WB Adaptive Multi-Rate Wideband 

AMR-WB+ Extended Adaptive Multi-Rate Wideband 

CELP Code Excited Linear Prediction 

ODG Objective Difference Grade 

PEAQ Perceptual Evaluation of Audio Quality 

TCX Transform coded excitation 

WB Wideband 
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General 



Test procedures and test sequences are necessary to test for correct implementations of the Extended Adaptive 
Multi-Rate Wideband (AMR-WB+) codec. 

Clause 5 explains the procedure for conformance testing and the detailed requirements. 

Annex A and B explain the digital test sequences and scripts to be executed for conformance testing of AMR-WB+ 
codec implementations. The test sequences and scripts are attached to the present document. 



5 Conformance 

Conformance testing of fixed-point encoder and decoder is done by showing bit-exactness to the fixed-point reference 

C-code implementation (given in 3GPP TS 26.273 [3]) or by meeting a set of minimum objective performance 

requirements by means of objective measures or by subjective testing. 

The bit-exact approach should be preferred over applying objective and subjective measures if it can be achieved 

without undue penalty on computational complexity. 

Conformance may also be concluded by subjective tests, in which performance not worse than that of the reference 

codec (given in 3GPP TS 26.273 [3]) is achieved. Details are given in clause 5.3. 

Floating-point implementations of encoder or decoder should be done by utilizing (compiling) the reference 
floating-point source code in specification (see 3GPP TS 26.304 [2]). In addition, as the exact behaviour of executables 
derived from floating-point code may depend on the platform onto which it was compiled, for implementations used in 
mobile equipment the objective measures shall be used for verification that minimum objective performance 
requirements are met. For floating-point implementations, which are not used in mobile equipment the usage of the 
objective measures is recommended. 

Conformance may also be concluded by subjective tests, in which performance not worse than that of the reference 
codec (see 3GPP TS 26.304 [2]) is achieved. Details are given in clause 5.3. 

The minimum performance requirements (objective and subjective measures) are the same for all implementations 
(fixed- and floating-point). 

If an implementor chooses to implement only mono encoder functionality, then conformance of only this functionality 
shall be tested. As above, conformance shall be shown by applying the bit-exact approach (fixed-point encoders), or by 
applying objective measures, or by subjective testing. 

Although the details of encoder and decoder implementation conformance testing depends on the comparison criteria 
given in sections 5.1 to 5.3, the following generic procedures applies for testing. 

A generic procedure for encoder implementation conformance testing is that the reference decoder decodes the output 
of implemented encoder and the resulting audio file is compared against the audio file got when the reference decoder 
decodes the output of reference encoder. 

A generic procedure for decoder implementation conformance testing is that the implemented decoder decodes the 
output of reference encoder and the resulting audio file is compared against the audio file got when the reference 
decoder decodes the output of reference encoder. 
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The conformance requirements are summarised in the below table. 





Fixed-point 


Floating-point 


Encoder* 


One of the following requirements must be met: 

bit-exactness to the fixed-point reference C- 
code (given in TS 26.273) shall be shown 
as explained in Clause 5.1 using the test 
sequences included in Annex A, 

or 

the set of minimum objective performance 
requirements shall be met as explained in 
Clause 5.2 using the test sequences 
included in Annex B, 

or 

the set of minimum subjective performance 
requirements shall be met as explained in 
Clause 5.3. 

The bit-exact approach should be preferred over 
applying objective or subjective measures if it 
can be achieved without undue penalty on 
computational complexity. 


The reference floating-point source code (given in 
TS 26.304) shall be used (compiled). 

In addition: 

For implementations used in mobile equipment: 

the set of minimum objective 
performance requirements shall be met 
as explained in Clause 5.2 using the test 
sequences included in Annex B, 

or 

the set of minimum subjective 
performance requirements shall be met 
as explained in Clause 5.3. 

For implementations not used in mobile 
equipment: 

the set of minimum objective 
performance requirements should be met 
as explained in Clause 5.2 using the test 
sequences included in Annex B, 

or 

the set of minimum subjective 
performance requirements should be met 
as explained in Clause 5.3. 


Decoder 


Same as for the fixed-point encoder 


Same as for the floating-point encoder 



*) If an implementor chooses to implement only mono encoder functionality, then conformance of only this functionality 
shall be tested. 



5.1 



Bit-exactness 



The implementor should choose the bit-exact approach for fixed-point encoder/decoder implementations. For 
fixed-point encoder and decoder implementations, test sequences are used for conformance testing. The test sequences 
consist of reference input audio files and corresponding reference encoder and decoder outputs. To meet the bit exact 
criterion all test sequences must give bit-exact result to the reference fixed-point C-code of 3GPP TS 26.273 [3]. The 
procedure is to run the encoder and decoder and check that the output (bit stream from encoder and decoded audio) is 
identical to the reference. Decoder only implementation conformance is checked using the reference encoder output and 
the reference decoder output test vectors. The test sequences are described in Annex A. 
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For the floating-point encoder and decoder implementations, executables should be derived by directly compiling the 
reference source code in the floating-point codec specification 3GPP TS 26.304 [2], Even though in that case the 
floating-point source code is identical to the reference code, different compilers and platforms may result in non 
bit-exact variations in the bit-streams produced by the encoder and decoder. Hence, in order to check for conformance 
of such implementations in mobile equipment the objective or subjective criteria given below shall be met. For other 
floating-point implementations, which are not used in mobile equipment the objective or subjective criteria given below 
should be tested and met. 



5.2 Objective quality measures 



Objective measures are used for testing the fulfilment of minimum performance requirements. These measures are the 
same for all implementations (fixed- and floating-point). Objective criteria used are PEAQ-ODG [4], wsegSNR and 
other statistics. 

5.2.1 Encoder 

Conformance of encoder implementations is tested with 'white-box' tests, which make use of specific structural 
knowledge of the AMR-WB+ codec. The tests verify the operation of various encoder parts, modules or modes such 
that when all tests are passed conformance can be concluded for the complete encoder implementation. Tests of the 
following codec parts/modules/modes must be passed in order to conclude conformance: 



Test* 


Mode 


Module/Part 


Reference 
configuration 


Criterion 


1.0-1.8 


MI=0..8;DTX disabled 


All 


26.273 encoder 
26.273 decoder 


All AMR-WB modes of 
AMR-WB+ are required 
to behave bit exactly to 
the FIP reference code 


2.0-2.8 


Ml=0..8; DTX enabled 


All 


26.273 encoder 
26.273 decoder 


All AMR-WB modes of 
AMR-WB+ are required 
to behave bit exactly to 
the FIP reference code 


3.0-3.2 


Ml = 23; 

ISF = 0.5, 1, 1.5; 

Mono operation 


ACELP 


26.304 encoder 


Weighted SNR 


4.0-4.2 


Ml = 23; 

ISF = 0.5, 1, 1.5; 

Mono operation 


TCX-256 


26.304 encoder 


Weighted SNR 


5.0-5.2 


Ml = 23; 

ISF = 0.5, 1, 1.5; 

Mono operation 


TCX-512 


26.304 encoder 


Weighted SNR 


6.0-6.2 


Ml = 23; 

ISF = 0.5, 1, 1.5; 

Mono operation 


TCX-1024 


26.304 encoder 


Weighted SNR 


7.0-7.3 


Ml = 23 

ISF = 0.5, 1.0, 1.5 

Mono operation 


ACELP/TCX 


26.304 encoder 


Weighted SNR 


8.0 


Ml = 23; 
ISF= 1; 

Mono operation 
Low complexity 
operation 


ACELP/TCX 
open-loop mode 
selection 


26.304 encoder 
26.273 decoder 


Relative number of 
identical mode selection 


9.0-9.2 


Ml = 26, 38, 47; 
ISF = 1.0; 
Stereo operation 


Low-band stereo 


26.304 encoder 
26.273 decoder 


PEAQ-ODG 


9.3-9.5 


Ml = 26, 38, 47; 
ISF = 1.0; 
Stereo operation 


Mid-band stereo 


26.304 encoder 
26.273 decoder 


PEAQ-ODG 


10.0 


Switched mode 
operation controlled 
with configuration file 


ACELP/TCX 
TCX/BWE/Stereo 


26.304 encoder 
26.273 decoder 


Weighted SNR 
PEAQ-ODG 


11 


Ml = 23 
ISF = 1.0 

Mono operation 


TCX+BWE 


26.304 encoder 
26.304 decoder 


PEAQ-ODG 



ETSI 



3GPP TS 26.274 version 6.1.0 Release 6 



ETSI TS 126 274 V6.1 .0 (2005-09) 



5.2.1.1 



AMR-WB mode compliance with DTX disabled 



All AMR-WB modes of AMR-WB+ are required to behave bit exactly to the FIP reference code. The implementation 
should comply with the test vectors given in 26.174. 



5.2.1.2 



AMR-WB mode compliance with DTX enabled 



All AMR-WB modes of AMR-WB+ are required to behave bit exactly to the FIP reference code. The implementation 
should comply with the test vectors given in 26.174. 



5.2.1.3 



ACELP mode compliance 



ACELP mode compliance is tested in an encoder configuration where the reference configuration is using the 
floating-point encoder according to 3GPP TS 26.304 [2]and the test configuration is composed of the encoder 
implementation to be verified. 

The test is run in forced-mode operation where the mode selection is disabled and the encoder is forced to ACELP 
mode. The compliance is tested by means of comparing the weighted segmental SNR (wsegSNR) resulting from both 
reference and test encoders. To that purpose, a modified instance of the reference encoder is used, which allows 
operation only in ACELP mode. This is accomplished by compiling the encoder applying a corresponding code patch 
(Annex B). Further, the function segsnr() is used to compute the wsegSNR. The test encoder implementation operated 
in a test mode, is modified in the same manner (forced to ACELP mode and enabled to compute wsegSNR). 

This setup is used for all items out of the test set (Annex B) and all configurations given in the following table. 



Test# 


Encoder command line 


3.0 


(Ml=23, ISF=0.5) 


3.1 


(Ml=23, ISF=1.0) 


3.2 


(Ml=23, ISF=1 .5) 



ACELP mode compliance can be concluded if in each of the test configurations in above table. 

• For no item a wsegSNR degradation of more than 2% is observed. 

• No more than 2% of frames shall have a wsegSNR more than 4 dB lower than the reference. 



5.2.1.4 



TCX-256 mode compliance 



TCX-256 mode compliance is tested in an encoder configuration where the reference configuration is using the 
floating-point encoder according to 3GPP TS 26.304 [2]and the test configuration is composed of the fixed-point 
encoder implementation to be verified. 

The test is run in forced-mode operation where the mode selection is disabled and the encoder is forced to 
TCX-256 mode. The compliance is tested by means of comparing the weighted segmental SNR (wsegSNR) resulting 
from both reference and test encoders. To that purpose, a modified instance of the reference encoder is used, which 
allows operation only in TCX-256 mode. This is accomplished by compiling the encoder applying a corresponding 
code patch (Annex B). Further, the function segsnr() is used to compute the wsegSNR. The test encoder implementation 
operated in a test mode, is modified in the same manner (forced to TCX-256 mode and enabled to compute wsegSNR). 

This setup is used for all items out of the test set (Annex B) and all configurations given in the following table. 



Test# 


Encoder command line 


4.0 


(Ml=23, ISF=0.5) 


4.1 


(Ml=23, ISF=1.0) 


4.2 


(Ml=23, ISF=1 .5) 



TCX-256 mode compliance can be concluded if in each of the test configurations in above table 

• For no item a wsegSNR degradation of more than 1 % is observed. 

• No more than 2% of frames shall have a wsegSNR more than 4 dB lower than the reference. 
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5.2.1.5 



TCX-512 mode compliance 



TCX-512 mode compliance is tested in an encoder configuration where the reference configuration is using the 
floating-point encoder according to 3GPP TS 26.304 [2]and the test configuration is composed of the fixed-point 
encoder implementation to be verified. 

The test is run in forced-mode operation where the mode selection is disabled and the encoder is forced to TCX-512 
mode. The compliance is tested by means of comparing the weighted segmental SNR (wsegSNR) resulting from both 
reference and test encoders. To that purpose, a modified instance of the reference encoder is used, which allows 
operation only in TCX-512 mode. This is accomplished by compiling the encoder applying a corresponding code patch 
(Annex B). Further, the function segsnr() is used to compute the wsegSNR. The test encoder implementation operated 
in a test mode, is modified in the same manner (forced to TCX-512 mode and enabled to compute wsegSNR). 

This setup is used for all items out of the test set (Annex B) and all configurations given in the following table. 



Test* 


Encoder command line 


5.0 


(Ml=23, ISF=0.5) 


5.1 


(Ml=23, ISF=1.0) 


5.2 


(Ml=23, ISF=1.5) 



TCX-512 mode compliance can be concluded if in each of the test configurations in above table. 

• For no item a wsegSNR degradation of more than 1 % is observed. 

• No more than 2% of frames shall have a wsegSNR more than 4 dB lower than the reference. 



5.2.1.6 



TCX-1024 mode compliance 



TCX-1024 mode compliance is tested in an encoder configuration where the reference configuration is using the 
floating-point encoder according to 3GPP TS 26.304 [2] and the test configuration is composed of the encoder 
implementation to be verified. 

The test is run in forced-mode operation where the mode selection is disabled and the encoder is forced to TCX-1024 
mode. The compliance is tested by means of comparing the weighted segmental SNR (wsegSNR) resulting from both 
reference and test encoders. To that purpose, a modified instance of the reference encoder is used, which allows 
operation only in TCX-1024 mode. This is accomplished by compiling the encoder applying a corresponding code 
patch (Annex B). Further, the function segsnr() is used to compute the wsegSNR. The test encoder implementation 
operated in a test mode, is modified in the same manner (forced to TCX-1024 mode and enabled to compute 
wsegSNR). 

This setup is used for all items out of the test set (Annex B) and all configurations given in the following table. 



Test# 


Encoder command line 


6.0 


(Ml=23, ISF=0.5) 


6.1 


(Ml=23, ISF=1.0) 


6.2 


(Ml=23, ISF=1.5) 



TCX-1024 mode compliance can be concluded if in each of the test configurations in above table. 

• For no item a wsegSNR degradation of more than 1 % is observed. 

• No more than 2% of frames shall have a wsegSNR more than 4 dB lower than the reference. 



5.2.1.7 



ACELP/TCX switching compliance 



Compliance of switching between ACELP and TCX modes is tested in an encoder configuration where the reference 
configuration is using the floating-point encoder according to 3GPP TS 26.304 [2]and the test configuration is 
composed of the fixed-point encoder implementation to be verified. 
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The test is run in free-mode operation where the closed-loop mode selection is is used. The compliance is tested by 
means of comparing the weighted segmental SNR (wsegSNR) resulting from both reference and test encoders. To that 
purpose, a modified instance of the reference encoder is used, which allows the computation of the wsegSNR. This is 
accomplished by compiling the encoder applying a corresponding code patch (Annex B). The test encoder 
implementation operated in a test mode, which allows forcing the mode usage to the saved mode selections of the 
reference encoder and computing the wsegSNR. 

This setup is used for all items out of the test set (Annex B) and all configurations given in the following table. 



Test* 


Encoder command line 


7.0 


(Ml=23, ISF=0.5) 


7.1 


(Ml=23, ISF=1.0) 


7.2 


(Ml=23, ISF=1 .5) 



ACELP/TCX switching compliance can be concluded if in each of the test configurations in above table 

• For no item a wsegSNR degradation of more than 1 % is observed. 

• No more than 2% of frames shall have a wsegSNR more than 4 dB lower than the reference. 



5.2.1.8 



ACELP/TCX open-loop mode selection compliance 



Correct ACELP/TCX selection for the signal content (speech vs. music) has a major impact in overall quality. 
Therefore, the algorithm selection has a specific conformance rule. The open-loop classification of the fixed-point 
encoder is tested comparing the ACELP/TCX selection. The reference source code in 3GPP TS 26.273 [3]contains 
functionality to output the coding mode selection. The MODE_SELECTION_CONFORMANCE needs to be defined in 
'cod_main_p_fx.c'. The outcome is that the reference encoder produces a file called 'mode_selection.dat.' The file 
contains in ASCII format the ACELP/TCX selection (ACELP = and TCX = 1). 

The conformance criterion P is calculated as follows: 

N (\ I 

N- I [\S (0-5,(0 

p_ '=1 

N 

Where N is the number of frames, S (/) and S ■ (i) are the ACELP/TCX selection output vector for reference and 
implemented encoder, respectively. 

Mode selection outputs are created running the reference and implemented fixed-point encoders with the test vectors. 
Since the bit-rate or the number of channels does not affect the open-loop classification, a test using only mono 
encoding with one bit rate is sufficient. 

This setup is used for all items out of the test set (Annex B) and the configuration given in the following table. 



Test# 


Encoder command line 


8.0 


(Ml=23, ISF=1.0) 



The fixed-point encoder open-loop classification is compliant to the specification when 0.99 < P < 1.00 . 



5.2.1.9 



Stereo operation compliance 



Stereo operation conformance is tested in an encoder - decoder configuration where the reference configuration is using 
the floating-point encoder and the fixed-point decoder according to 3GPP TS 26.304 [2]and 3GPP TS 26.273 [3], 
respectively. The test configuration (codec in test) is composed of the encoder in test (EiT) and the fixed-point decoder 
according to 3GPP TS 26.273 [3]. 
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Two sets of tests of the stereo operation are defined using the described configuration, one testing the low-band stereo 
operation and one testing the mid-band stereo. Common for these tests is that for the codec in test they apply an 
approach where the bit streams of the reference encoder and the encoder in test are merged before decoding. 
Specifically, the part of the bit stream generated by EiT belonging to the specific operation is merged into the bit stream 
of the reference encoder, thereby replacing the corresponding part generated by the reference encoder. The output files 
produced by reference configuration and such way composed test configurations are then compared using PEAQ [4]. 

For the test of the low-band stereo the bit stream is composed by using the bit groups corresponding to low-band stereo 
from the EiT and the rest from the reference encoder. 

This setup is used for all items out of the test set (Annex B) and all configurations given in the following table. 



Test* 


Encoder command line 


9.0 


(Ml=26, ISF=1.0) 


9.1 


(Ml=38, ISF=1.0) 


9.2 


(Ml=47, ISF=1.0) 



Low-band stereo operation compliance can be concluded if in each of the test configurations in above table. 

• For no item a PEAQ-ODG deviation of less than -0.2 is observed. 

• The mean of the PEAQ-ODG deviation scores is not below -0.05. 

For the test of the mid-band stereo the bit stream is composed by using the bit groups corresponding to mid-band stereo 
from the EiT and the rest from the reference encoder. In addition EiT operation must be modified such that the EiT uses 
the states of the anti -dithering logic of the reference encoder. This is accomplished by compiling the encoder applying a 
corresponding code patch (Annex B). 

This setup is used for all items out of the test set (Annex B) and all configurations given in the following table. 



Test# 


Encoder command line 


9.3 


(Ml=26, ISF=1.0) 


9.4 


(Ml=38, ISF=1.0) 


9.5 


(Ml=47, ISF=1.0) 



Mid-band stereo operation compliance can be concluded if in each of the test configurations in above table. 

• For no item a PEAQ-ODG deviation of less than -0.2 is observed. 

• The mean of the PEAQ-ODG deviation scores is not below -0.05. 

5.2.1 .10 Mode switching operation compliance 

Compliance of switching the bit rate and ISF is tested in an encoder configuration where the reference configuration is 
using the floating-point encoder according to 3GPP TS 26.304 [2]and the test configuration is composed of the 
fixed-point encoder implementation to be verified. 

The test is run in an operation using three configuration files in mono operation where in the first configuration the ISF 
is change from 0.5 to 1.5 at fixed FT=23, in the second configuration the FT is changed from 16 to 23 at fixed ISF=1.0, 
and the third configuration both FT and ISF are changed. The compliance is tested by means of comparing the weighted 
segmental SNR (wsegSNR) resulting from both reference and test encoders. To that purpose, a modified instance of the 
reference encoder is used, which allows the computation of the wsegSNR. This is accomplished by compiling the 
encoder applying a corresponding code patch (Annex B). The test encoder implementation should be patched similarly 
to allow computing the wsegSNR. 

This setup is used for all items out of the test set (Annex B) and all configurations given in the following table. 



Test* 


Encoder command line 


10.0 


Config file switch fs.txt 


10.1 


Config file switch mode.txt 


10.2 


Config file switch_allcat.txt 
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Switching compliance can be concluded if in each of the test configurations in above table. 
• For no item a wsegSNR degradation of more than 1 % is observed. 



5.2.1.11 



Overall mono compliance 



Mono operation compliance (including bandwidth extension) is tested in an encoder - decoder configuration where the 
reference configuration is using the floating-point encoder and the floating-point decoder according to 
3GPP TS 26.304 [2]. The test configuration is composed of the fixed-point encoder implementation to be verified and 
the floating-point decoder according to 3GPP TS 26.304 [2]. 

In order to make the testing independent of possible deviations of ACELP/TCX it is run in forced-mode operation. To 
that purpose, a modified instance of the reference encoder is used, which merely operates in TCX modes and which 
prohibit ACELP mode use, and which allows for tracing TCX mode selection. This is accomplished by compiling the 
encoder applying a corresponding code patch (Annex B). The test encoder implementation should be operable in a test 
mode, which allows forcing the TCX mode usage to the saved mode selections of the reference encoder. The output 
files produced in reference and test configuration are then both compared to the original input signal by means of 
measuring the PEAQ ODG. Then the absolute value of the difference of both PEAQ ODGs is computed and used for 
checking compliance. 

This setup is used for all items out of the test set (Annex B) and the configuration given in the following table. 



Test# 


Encoder command line 


11.0 


(Ml=23, ISF=1.0) 



Overall mono compliance can be concluded if in the test configuration in above table 

• For no item an absolute PEAQ-ODG difference of more than -0. 1 is observed 

• The mean of the PEAQ-ODG difference scores is not greater than 0.01 . 

5.2.2 Decoder 

Conformance of decoder implementations is tested with 'black-box' tests. Such tests verify the output of the given 
decoder implementation against the output of the reference decoder (see 3GPP TS 26.273 [3]) for certain input test 
vectors and various codec operation modes and bit rates. The following tests must be passed in order to conclude 
conformance: 



Test* 


Encoder operation 


Decoder condition 


Reference 
configuration 


Criterion 


1.0-1.8 


Ml=0..8; DTX disabled 


Without frame 
erasures 


26.304 encoder 
26.273 decoder 


All AMR-WB modes of 
AMR-WB+ are 
required to behave bit 
exactly to the FIP 
reference code 


2.0-2.8 


Ml=0..8; DTX enabled 


Without frame 
erasures 


26.304 encoder 
26.273 decoder 


All AMR-WB modes of 
AMR-WB+ are 
required to behave bit 
exactly to the FIP 
reference code 


3.0-3.7 


All mono rates, 
Various ISFs 


Without frame 
erasures 


26.304 encoder 
26.273 decoder 


PEAQ-ODG 


4.0-4.7 


All mono rates, 
Various ISFs 


With frame erasures 


26.304 encoder 
26.273 decoder 


PEAQ-ODG 


5.0 - 5.2 


Rate = 14, 24, 48 kbps 
Stereo 


Without frame 
erasures 


26.304 encoder 
26.273 decoder 


PEAQ-ODG 


6.0-6.2 


Rate = 14, 24, 48 kbps 
Stereo 


With frame erasures 


26.304 encoder 
26.273 decoder 


PEAQ-ODG 


7.0-7.2 


Rate = 14, 24, 48 kbps 
Stereo 


Without frame 
erasures; 
Mono output 


26.304 encoder 
26.273 decoder 


PEAQ-ODG 


8.0-8.2 


Rate = 14, 24, 48 kbps 
Stereo 


With frame erasures; 
Mono output 


26.304 encoder 
26.273 decoder 


PEAQ-ODG 
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5.2.2.1 



AMR-WB mode compliance with DTX disabled 



All AMR-WB modes of AMR-WB+ are required to behave bit exactly to the FIP reference code. The implementation 
shall comply with the test vectors given in 3GPP TS 26.174 [6]. 



5.2.2.2 



AMR-WB mode compliance with DTX enabled 



All AMR-WB modes of AMR-WB+ are required to behave bit exactly to the FIP reference code. The implementation 
shall comply with the test vectors given in 3GPP TS 26.174 [6]. 



5.2.2.3 



Extension mode compliance in mono operation without frame erasures 



Compliance is tested in an encoder - decoder configuration where the reference configuration is using the floating-point 
encoder and the fixed-point decoder according to 3GPP TS 26.304 [2] and 3GPP TS 26.273 [3], respectively. The test 
configuration is composed of the floating-point encoder according to 3GPP TS 26.304 [2] and the fixed-point decoder 
implementation to be verified. 

The output files produced in reference and test configuration are compared using PEAQ. 

The following table list the tests to be carried out for all items of the test set (Annex B). 



Test# 


Encoder command line 
options 


Decoder command line 
options 


3.0 


-mi 19 -isf 0.9375 


None 


3.1 


-mi 20 -isf 1 


None 


3.2 


-mi 21 -isf 1.125 


None 


3.3 


-mi 22 -isf 1.33333 


None 


3.4 


-mi 23 -isf 1 .5 


None 



Compliance can be concluded if in each of the tests in above table. 

• For no item a PEAQ-ODG deviation of less than -0.2 is observed. 

• The mean of the PEAQ-ODG deviation scores is not below -0.05. 



5.2.2.4 



Extension mode compliance in mono operation with frame erasures 



Compliance is tested in an encoder - decoder configuration where the reference configuration is using the floating-point 
encoder and the fixed-point decoder according to 3GPP TS 26.304 [2] and 3GPP TS 26.273 [3], respectively. The test 
configuration is composed of the floating-point encoder according to 3GPP TS 26.304 [2] and the fixed-point decoder 
implementation to be verified. 

The output files produced in reference and test configuration are compared using PEAQ. 

The following table list the tests to be carried out for all items of the test vector set (TBA). 



Test# 


Encoder command line 
options 


Decoder command line 
options 


4.0 


-mi 19 -isf 0.9375 


None 


4.1 


-mi 20 -isf 1 


None 


4.2 


-mi 21 -isf 1.125 


None 


4.3 


-mi 22 -isf 1.33333 


None 


4.4 


-mi 23 -isf 1 .5 


None 
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The fertile to be used is part of the test vector set (Annex B) and simulates a random frame loss of 10 %. 
Compliance can be concluded if in each of the tests in above table. 

• For no item a PEAQ-ODG deviation of less than -0.2 is observed. 

• The mean of the PEAQ-ODG deviation scores is not below -0.05. 



5.2.2.5 



Extension mode compliance in stereo operation without frame erasures 



Compliance is tested in an encoder - decoder configuration where the reference configuration is using the floating-point 
encoder and the fixed-point decoder according to 3GPP TS 26.304 [2] and 3GPP TS 26.273 [3], respectively. The test 
configuration is composed of the floating-point encoder according to 3GPP TS 26.304 [2] and the fixed-point decoder 
implementation to be verified. 

The output files produced in reference and test configuration are compared using PEAQ. 

The following table list the tests to be carried out for all items of the test set (Annex B). 



Test* 


Encoder command line 
options 


Decoder command line 
options 


5.0 


-rate 1 4 


None 


5.1 


-rate 24 


None 


5.2 


-rate 48 


None 



Compliance can be concluded if in each of the tests in above table. 

• For no item a PEAQ-ODG deviation of less than -0.2 is observed. 

• The mean of the PEAQ-ODG deviation scores is not below -0.05. 



5.2.2.6 



Extension mode compliance in stereo operation with frame erasures 



Compliance is tested in an encoder - decoder configuration where the reference configuration is using the floating-point 
encoder and the fixed-point decoder according to 3GPP TS 26.304 [2] and 3GPP TS 26.273 [3], respectively. The test 
configuration is composed of the floating-point encoder according to 3GPP TS 26.304 [2] and the fixed-point decoder 
implementation to be verified. 

The output files produced in reference and test configuration are compared using PEAQ. 

The following table list the tests to be carried out for all items of the test vector set (Annex B). 



Test# 


Encoder command line 
options 


Decoder command line 
options 


6.0 


-rate 12 


-fer fertile 


6.1 


-rate 24 


-fer fertile 


6.2 


-rate 36 


-fer fertile 



The ferfile to be used is part of the test vector set (Annex B) and simulates a random frame loss of 10%. 
Compliance can be concluded if in each of the tests in above table. 

• For no item a PEAQ-ODG deviation of less than -0.2 is observed. 

• The mean of the PEAQ-ODG deviation scores is not below -0.075. 

5.2.2.7 Extension mode compliance in stereo to mono downmixing operation without 

frame erasures 

Compliance is tested in an encoder - decoder configuration where the reference configuration is using the floating-point 
encoder and the fixed-point decoder according to 3GPP TS 26.304 [2] and 3GPP TS 26.273 [3], respectively. The test 
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configuration is composed of the floating-point encoder according to 3GPP TS 26.304 [2] and the fixed-point decoder 
implementation to be verified. 

The output files produced in reference and test configuration are compared using PEAQ. 

The following table list the tests to be carried out for all items of the test set (Annex B). 



Test* 


Encoder command line 
options 


Decoder command line 
options 


7.0 


-rate 1 4 


-mono 


7.1 


-rate 24 


-mono 


7.2 


-rate 48 


-mono 



Compliance can be concluded if in each of the tests in above table. 

• For no item a PEAQ-ODG deviation of less than -0.2 is observed. 

• The mean of the PEAQ-ODG deviation scores is not below -0.05. 

5.2.2.8 Extension mode compliance in stereo to mono downmixing operation with 

frame erasures 

Compliance is tested in an encoder - decoder configuration where the reference configuration is using the floating-point 
encoder and the fixed-point decoder according to 3GPP TS 26.304 [2] and 3GPP TS 26.273 [3], respectively. The test 
configuration is composed of the floating-point encoder according to 3GPP TS 26.304 [2] and the fixed-point decoder 
implementation to be verified. 

The output files produced in reference and test configuration are compared using PEAQ. 

The following table list the tests to be carried out for all items of the test vector set (Annex B). 



Test* 


Encoder command line 
options 


Decoder command line 
options 


8.0 


-rate 12 


-fer fertile -mono 


8.1 


-rate 24 


-fer fertile -mono 


8.2 


-rate 36 


-fer fertile -mono 



The fertile to be used is part of the test vector set (Annex B) an simulates a random frame loss of 10%. 
Compliance can be concluded if in each of the tests in above table. 

• For no item a PEAQ-ODG deviation of less than -0.2 is observed. 

• The mean of the PEAQ-ODG deviation scores is not below -0.05. 

5.2.3 Additional objective criteria (valid for both fixed- and floating point 
versions) 

• If the mobile equipment requires 16 kHz or 8 kHz output sampling rate, then the decoder implementation shall 
be able to provide 16 or 8 kHz output sampling rate, respectively. 

• If the mobile equipment requires mono output, then the decoder implementation shall be able to provide mono 
output when the input bit stream is stereo. Conformance is verified applying the criteria given in clauses 5.2.2.7 
and 5.2.2.8. 

• The delay of implementations for decoders and encoders used in mobile equipment shall not exceed the 
reference decoder and encoder delay. 

• Decoder implementation shall support all the modes and bit rates described in the specifications. 

Encoder shall be able to perform switching between AMR-WB and extension modes when operated at 16 kHz input 
sampling rate. Conformance specification contains test vectors and mode information file to check the functionality. 
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Fixed-point and floating-point AMR-WB modes as well as VAD/DTX operation compliance is the same to 
3GPP TS 26.173 [3]and 3GPP TS 26.204 [5], respectively. 

5.3 Subjective conformance testing 

Conformance may be concluded by subjective testing in which in no test condition performance worse than the 
reference codec (see 3GPP TS 26.273 [3]) is proven. Test plan: (characterization test plan tba when finalized). 

Subjective conformance testing is allowable for all extension modes. AMR-WB modes of AMR-WB+ shall conform 
according to respective AMR-WB specifications. 
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Annex A (normative): Codec test sequences for bit exact 
testing of AM R-WB+ 

This Annex describes the test sequences designed to exercise the bitexactness of a fixed-point implementation of the 
extended adaptive multi-rate wideband (AMR-WB+) codec (see 3GPP TS 26.290 [1]). 

The companion archive 'bitexact.zip' contains test items (original wav files), reference encoder and decoder executables 
and scripts to generate all required test sequences that are needed for testing bit exactness according to clause 5.1. 
Further, for testing the bitexactness of the decoder, test bitstream files are provided. 
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Annex B (normative): Codec test sequences for objective 
testing of AM R-WB+ 

This Annex describes the test sequences designed to exercise the objective testing of a non-bit exact implementation of 
the extended adaptive multi-rate wideband (AMR-WB+) codec (see 3GPP TS 26.290 [1]). 

The companion archive 'objective.zip' contains: 

• test items (original wav files); 

• reference encoder and decoder executables; 

• scripts applying the verification procedures; and 

• code patches to the reference codec to enable certain objective tests. 
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Annex C (informative): 
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